The present invention relates to computer vision systems. More particularly, the present invention relates to an improved apparatus and method for determining the characteristics of an articulated member.
Computer vision is a development in the broader field of computer systems. One of the goals of computer vision systems is to recognize objects from electronic images. For example, a video camera may record a single image or a series of images and relay the information to a computer system. The computer system may then be able to determine the relative position of objects, persons, animals or any other images within the computer image.
Several types of applications exist for computer vision systems. By way of example, in industrial applications, a computer vision system may be utilized to monitor and control a robotic arm used in a fabrication system. Computer vision systems may also be used for pattern recognition purposes, manufacturing quality control, motion analysis and security.
FIG. 1 illustrates a typical scene 2 that a computer vision system may be used to analyze. Scene 2 may include a subject 4 and a background 5. Subject 4 can be any object which the computer vision system may be used to analyze. By way of example, in the illustrated example, subject 4 is a person. Background 5 may include any part of scene 2 that is not part of subject 4. In the illustrated example, scene 5 includes the sun, clouds, the sky and hills.
FIG. 2 illustrates an electronic image 10 derived from scene 2 of FIG. 1. Electronic image 10 may include an electronic representation of the background 25, and an electronic representation of subject 4, subject image 20. Typically, computer vision systems are concerned with various aspects of subject 4 by way of subject image 20. In the illustrated example, several features of subject 20 are enumerated 20a through 20h. By way of example, the enumerated features may include a center of body point 20a, a body box 20b, a head box 20c, an arm box 20d, a forearm box 20e, a hand point 20f, an elbow point 20f, and a shoulder point 20h. However, in general, any type of suitable characteristic of subject 20 may be desired to be obtained through the computer vision system.
Another focus of computer vision systems has been on the ability to interact with electronic image 10. By way of example, computer vision systems may be used to control a machine based upon the movements or position of subject 4. Alternately, electronic image 10 and subject image 20 may be may be manipulated by subject 4 through movement.
Real time interaction has also been attempted by prior art systems. However, interaction between subject 4 and the computer vision system may also run into the limitations of precise subject characterization and real-time operations. Prior art systems that may have allowed real-time interaction between a subject and a computer vision system typically require large amounts of computational power.
As will be discussed in more detail below, the prior art methods of determining the characteristics of subject image 20, are often times complex and unable to be accomplished in real time. For example, one method of subject characterization is through the use of stereo vision. In stereo vision, two cameras are used to capture the electronic images of subject 4. One advantage of the stereo vision method is that a 2-D or a 3-D image may be generated.
However, problems exist with registration in systems which use stereo vision. That is, it is often difficult to match the two images captured by the separate video cameras to create a coherent single image. The registration problem, often leads to faulty characterization of subject image 20. Additionally, the use of two cameras adds to the expense and complexity of the computer vision system. Further, stereo vision systems are typically not capable of characterizing subject image 20 in real time.
Another method of subject characterization is through the use of markers. Typically, markers are placed at various points on a subject that the computer vision system would like to detect. In the illustrated example, markers may be placed on shoulder point 20h, elbow point 20g, hand point 20f, and center point 20a. The marker system may allow for real time characterization of subject image 20, however, there are several disadvantages.
The primary disadvantage associated with markers is that markers must be placed on subject 4 of FIG. 1 in order to obtain the various characteristics of subject image 20. The physical placement of markers is often not possible in situations where subject characterization is desired of objects not suitable for marking. For example, subject characterization may be desired of an animal in the wild. In such a case, markers cannot be placed on the wild animal.
Another disadvantage lies in the fact that subject 4 must physically wear the markers. There may be a number of situations where the subject is not capable or does not desire to wear markers, therefore, making subject characterization difficult. Additionally, the marker system may not be capable of obtaining all of the desired characteristics of subject 4. For example, body box 20b, head box 20a, arm box 20d, and forearm box 20e are two-dimensional shapes which may require a large number of markers in order to adequately characterize through computer vision. Once again, increasing the number of markers only adds to the difficulty and burden of using the marker system.
Another method of subject characterization using computer vision is to perform pretraining. A computer vision system may be used to preliminarily take and analyze electronic images of subject 4 before the actual subject characterization takes place in a real life situation. In this method, the computer vision system may be able to more accurately characterize subject image 20 in a real life situation. However, this assumes that subject 4 is available for pretraining. If subject 4 is a wild animal or another subject that is not available for pretraining, this method of subject characterization may not be feasible. Further, the change in conditions between the training setting and the real life setting may diminish any benefit derived from pretraining.
This assumes that the prior art systems are even capable of accurately capturing a subject image. To further distinguish the different points or parts of subject image 20 prior art systems may repetitively compare the current subject image 20 with all the possible sizes, shapes and positions subject 4 may be in. Typically, this exhaustive approach is generally incapable of producing real-time results.
As discussed, prior art systems, including the marker system and the pretraining system, may not be feasible in real life situations. However, other systems that do not utilize markers or pretraining may not capable of real time operations because of the extensive amount of computations needed to accomplish subject characterization. Typically, prior art systems use exhaustive algorithms to determine the desired characteristics of subject image 20.
Some prior art computer vision systems have been capable of interaction between subject 4 and electronic image 10. However, real time interaction between subject 4 and the computer vision system have continued to run into the problem of requiring extremely large amounts of computational power.
Thus, what is desired is a more efficient interactive computer vision system. A computer vision system that is capable of providing real-time interaction with the computational power of common computer systems is further desired.